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Abstract 


Ensembles of networks are used as null-models to discriminate network structures. We 
present an efficient algorithm, based on the maximal entropy method to generate network 
ensembles defined by the degree sequence and the rich-club coefficient. The method is 
applicable for unweighted, undirected networks. The ensembles are used to generate 
correlated and uncorrelated null-models of a real networks. These ensembles can be 
used to define the partition of a network into soft communities. 


1 Introduction 

Attention has recently been directed to the generation of network ensembles as they 
are used to identify statistically significant patterns on real networks [TJ [2, [3], The 
ensemble is used as a reference null-model for comparison purposes. The definition of 
communities is an example of a pattern defined using a null-model. The communities 
are defined as the groups of nodes where the number of links between the communities 
is smaller than expected [3]. This ‘expected number of links between the communities’ 
is evaluated using a null-model. A well defined null-model should capture the essence of 
the network’s topology without introducing spurious or unwanted structural properties. 
The most common criterion is to define a null-model that has the same degree sequence 
as the real network and any other topological property has a nonspecific structure. This 
criterion captures the basic topological composition of a network but in some cases it 
cannot remove higher topological properties due to structural constraints EE]. 

Shannon’s entropy can be used to describe our state of knowledge of the network struc¬ 
ture 0IBE- The maximisation of this entropy can be used to create an ensemble 
that is ‘maximally non-committal’, that is given some constraints, the ensemble is as 
unbiased as it is possible. These unbiased network ensembles can be evaluated via 
the Maximum Entropy method (MaxEnt) [H J)'. [TU, 21 E] ° 1- the Maximum Likelihood 
method la na na nn, which are equivalent. The generation of null-models using 
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these information-based methods is attractive, as they provide analytical expressions 
to describe the basic properties of the ensemble, and in general, can be computed effi¬ 
ciently. 

In this paper the main aim is to present an algorithm to create null-models defined by 
the conservation of the degree sequence and the rich-club connectivity. The networks 
considered here are undirected, unweighted and without self loops but multiple links 
between a pair of nodes are allowed. The algorithm is based on a previous result where 
the maximal entropy approach (MaxEnt) was used to generate the network ensemble. 
The method is attractive because the set of probabilities describing the ensemble can be 
evaluated very efficiently mi- We use the method to construct ensembles with similar 
correlations as the real network and also to construct ensembles with minimal correla¬ 
tions. As a sub-aim we present an example of how to combine these ensembles to define 
‘soft’ network-communities. 


2 Generation of the ensembles 


The null-model is obtained using the maximal entropy method with the constraints that 
the degree sequence and rich-club coefficient are conserved HU. The constraints are the 
sequences {Ay,..., k at} and {kf ,..., kf}, where N is the number of nodes. The first 
sequence contains the degree k of the nodes. The nodes are ranked in decreasing degree 
order, the node with the highest degree is ranked first and so on. The second sequence 
is the number of links kf that node r shares with nodes r' of higher rank, r' < r. The 
total number of links in the network is L = 1/2 k r = X(r=i kf. The ranked based 
rich-club coefficient m is <b r = (2/(r(r — 1))) Yfi= i kf, thus conserving kf is equivalent 
to the conservation of the rich-club coefficient. 


The sequence {A;^} is bounded kf < Ay, but kf could be large enough to allow multiple 
links between two nodes. If only one link can exist between two nodes then kf < r— 1. If 
kf > r — 1 implies that there are more than r — 1 links between node r and the r— 1 nodes 
of higher rank, which means the existence of at least one multiple link (Fig. 0a))- 

The Shannon entropy of the network is S' = — fff = i Y^f=ij^iPi,j logThe maximal 
entropy solution is given by the probabilities El 


PiJ 


w{i) (ki - kf ) kf 

Yfn=i w ( n ) {kn - kf) L 


( 1 ) 


where 

w(m) = (2) 

YT =1 w if){ki - kf) - kfufm - 1) 

The values of w(m) are defined recursively with the initial condition u>(l) = 1. The 
average number of links between nodes i and j is e,;j = Lpij with variance Sij = Lpij( 1 — 
Pij)- 
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(a) (b) 

Figure 1: Zachary karate club, (a) The degree {Ay} sequence (blue) and {£;+} sequence 
(green), k+ is always less or equal to Ay. The dotted line is the bound to restrict the 
ensemble connectivity to only one link per pair of nodes, (b) The {A;+} sequence for the 
null-model with the constraint that on average there is only one link between two nodes 
(green) and for the null model without this restriction (orange). 


By construction the ensemble satisfies the ‘soft’ constraints (Ay) = ^2jLi Lp r j = Ay and 
(k+) = X^'=i Lp r ,j = A;+, where the variance of the degree is cr 2 (Ay) = LJ2f^ r Pr,j( 1 — 
Pr,j)- 

We consider three different null-models all constructed using Eqs. Given the 

degree sequence {ki} the nulls are define by the following constraints: 

1. The sequence {kf } is given. We refer to this as the maximal-entropy model ME1. 

2. The sequence {k+} is not given. This sequence is obtained numerically by max¬ 
imising the entropy with the restrictions that k+ < Ay and k+ < r — 1. The last 
restriction is to impose that on average there is only one link between two nodes. 
We called this model ME2. 

3. The sequence {k+} is not given. This sequence is obtained numerically by max¬ 
imising the entropy with the restriction that kf < fcj. In this case it is possible to 
have, on average, more than one link between a pair of nodes but self-loops are 
not allowed. We called this model ME3. 

The algorithm to evaluate the second and third cases is in the Appendix. Fig. 0b) 
shows an example of the {k+} sequence found by our algorithm for the ME2 and ME3 
models for the Zachary club. 

It is known that the rich-club coefficient is related to the degree-degree correlation m- 
As one of the ME1 ensemble constraint is the conservation of the rich-club sequence, 
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the degree-degree correlation produced by the ensemble is similar to the degree-degree 
correlation of the real network mm- 

The ME2 removes the restriction that the ensemble should have the same rich-club 
sequence as the real network but still has the restriction that on average there should be 
only one link per pair of nodes. The consequence of this last restriction is that the ME2 
model will produce correlated ensembles if the maximal degree k m ax is greater than the 
cut-off degree k c = s/2 L [6l [5]. 


The ME3 removes the restriction that there is only one link per pair of nodes but self 
loops are not allowed. This restriction should produce decorrelated or almost decorre- 
lated networks [5]. 


To decide if the ensemble generates a biased null-model we use the average nearest 
neighbours degree given by ( k nn (k )) = Ylk' k'P(k'\k) jT8j , where P{k'\k) is the condi¬ 
tional probability that given a node with degree k its neighbour has degree k' . For an 
uncorrelated network (k an ) = (k 2 )/(k). In our case pi.j is known so 


(knn(k)) 


N 


N 


N k 


I i, ^^PijLkj I S ki ,k, 


i —1 


3 = 1 


(3) 


where 8 ki: k = 1 if ki = k and zero otherwise. 

Another approach to evaluate if the statistical properties of the ensemble are biased 
is via the coefficient of variation and the inverse participation ratio for the degree se¬ 
quence M- The coefficient of variation for the degree ki is 


c{ki) = 



The quantity we are interested is the term 

N ( N \ 2 

1 , 1 = », 

jA* \j^i ) 


(4) 


(5) 


which is an inverse participation ratio. This ratio measures the number of probability 
terms pij contributing effectively to Eq. Q. If only one probability term contributes, 
then Ij = 1. If all the probability terms contribute in equal measure then Ij = C for all 
i, where C is a constant. The participation ratio gives a measure of the homogeneity of 
the statistical properties evaluated from the ensemble. 


2.1 Results 

We compare the models presented here with two of the most common null-models, 
Newman-Girvan’s model [3] (NG) and restricted randomisation (RR). In Newman- 
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Girvan’s the probability that nodes i and j share a link is pij = kikj/(2L). As the proba¬ 
bility may exceed the value of 1, the model is applicable if kikj > 2 L or k m ax < V / 2L [5]. 
The restricted randomisation consist of reshuffling at random the links between the 
nodes, with the restriction that the degree of the nodes is conserved. If there is the 
extra restriction that two nodes cannot share more than one link, we refer to this model 
as the RR1 model. The RR1 model will produce correlated networks if the maximum 
degree is larger than the cut-off degree k max . If multiple links between two nodes are 
allowed but not self loops we called this model RR2. In this case is possible to have 
multiple links between nodes so we expect that the nulls produced by this model will 
tend to be almost uncorrelated. They would be some correlation as it is expected that 
the number of self-loops for a node of degree k is k 2 /((k)N) |T9], where (k) is the average 
degree, as in here self-loops are not allowed, then the null model could have some small 
bias in the correlations [5j. 

Fig. I shows the average neighbours degree ( k nn (k )) obtained form the data and the 
different null models for the co-authorship of High Energy Physics (Hep-Th) and the 
AS-Internet networks. The Hep-Th network was chosen as it is assortative. The AS- 
Internet was chosen because the maximum degree is larger than the cut-off degree. 

For the Hep-Th network the null model ME1 has a similar degree-degree correlation as 
the original network (Fig. [2^a)). If we remove the constraint that there is, on average, 
only one link between two nodes then, as we do not have a degree cut-off, both models 
ME2 and ME3 produce ensemble with similar degree-degree correlation (Fig. [2](b)). 
Which is also the case for the RR1, RR2 and NG models (Fig. [2](c)). 

For the Internet, as in the previous example, the ME1 generates an ensemble with very 
similar degree-degree correlation as the original network (Fig. [2](d)). However, this 
network has a degree cut-off ( k c ~ 260) implying that if that, on average, only one link 
per pair of nodes is allowed then there is a degree-degree correlation between nodes. 
This is the case for ME2 and RRl (Fig.[2](e)-(f)). There is a difference between the RR1 
and the ME2 methods. In the ME2 multiple links are not allowed, the maximisation of 
the entropy produces an ensemble that minimises the degree-degree correlation but, due 
to the cut-off degree restriction, it fails to decorrelate nodes of high degree. Compared 
this with the RRl, which produces an ensemble where there are correlations at all 
degrees. 

For the case that we allow more than one link between two nodes, the two models MM3 
and RR2 produce almost uncorrelated networks. There still some correlation at high 
degrees, this correlation is due to restriction that self-loops are not allowed. We do not 
present the results for the NG model as k max > V^L- 

Fig. i shows the inverse participation ratio (Eq. [5]) for the Hep-Th and AS Internet 
for the ME1, ME2 and ME3 ensembles. Similarly as the results obtained using the 
average neighbours degree, the inverse participation ratio distinguishes if the ensemble 
produces correlated networks. In both networks the ME1 ensemble has a widely varying 
inverse participation ratio as by construction the ensemble is correlated. For the Hep-Th 
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(d) 


(e) 


(f) 


Figure 2: Comparison of the average neighbours degree for the Hep-Th and AS Internet 
networks and their null-models. The original network and the null ME1 (left column). 
The entropy maximisation nulls ME2 and ME3 (middle column). Newman-Girvan (NG), 
randomisation with (RR1) and without constraints (RR2) in the number of links between 
nodes (right column). The dashed line shows the value of ( k nn ) = (k 2 )/(k) which 
corresponds to a decorrelated network. 
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(a) (b) 

Figure 3: Inverse participation ratio for the (a) High Energy Physics co-authors net¬ 
works and the (b) AS Internet for the different MaxEnt ensembles. 


network the probabilities Pi.j obtained from ME2 and ME3 are such that terms Pi : j 
are almost homogenous, this is expected as by construction the MaxEnt method produces 
probabilities pi.j that are maximally neutral. This is not the case for the AS Internet, the 
existence of a cut-off degree and the constraints of no multiple links (ME2) or no self¬ 
loops (ME3) shows that the MaxEnt solutions, ‘push’ towards homogenous contributions 
in the evaluation of Yli=iPi,j but this is not achieved due to the structural constraints. 
Nevertheless, this departure from homogeneity can be use to define the cut-off degree, 
in this case when the inverse participation ratio shows that the probability terms stop 
contributing in equal measure. For the case of the AS Internet, this happens k cu t = 102 
for ME2 and k cu t = 1042 for ME3. 

3 Communities and the ensembles 

A common use of null models is in the discovery of network communities [3j. The 
communities are obtained by maximising the modularity function Q. If a^- is an entry 
of the adjacency matrix and is the expected number of links between nodes i and j 
then the modularity is 

N N 

Q = ^2^2 “ e ij) s (si,gj) (6) 

* 3 

where gi denotes the community that node i belongs and 5(x, y) = 1 if x = y and zero 
otherwise. The most common null-model used in modularity maximisation is Newman- 
Girvan model, where e t j = kikj/(2L). 

Squartini and Garlaschelli [3] argued that there are several disadvantages using Newman- 
Girvan’s null model in the discovery of communities via the modularity function, the 
main limitation is that the NG model is only feasible for networks where the maximum 
degree satisfies k max < V%L- They proposed that a better approach is to use an e^ 
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obtained using a maximum-likelihood (maximal entropy) method. Here we follow their 
suggestion and consider e t j = Lpij where the probability p tJ can be obtained from one 
of the null-models presented here. 

There are many techniques to obtain the maximisation of the modularity function Q [2D] • 
Here we evaluate the community structure using the spectral method [21j. This method 
partitions the network into two subnetworks. Then each subnetwork is partition again 
into two parts, and so on. This method creates a hierarchical partition of the network 
that can be represented with a dendrogram. The partition stops when the modularity is 
maximal or if it is not possible to split the network any further. Usually this method is 
stopped when the modularity is maximal but it is known that this does not imply that we 
obtain the best community structure as some networks have high modularity but poor 
community structure and other networks have low modularity and good community 
structure [22;; [23] • We would apply the spectral method to our networks until is not 
possible to partition them any further, regardless if the modularity is maximal. 

It is also known that the modularity methods are not deterministic as the communities 
obtained depend on the initial conditions used when starting the procedure [211I2SU2S]- 
To take this dependancy into consideration we randomised the ranking of the nodes of 
equal degree. We evaluate the communities using 100 networks with randomised ranks 
and then we evaluate how many times a pair of nodes appear in the same community. 
The set of nodes that always appear in pairs and share a link are considered the core 
of the communities as they are independent of the ranking order and they must play a 
more relevant role in the structure of the community [24]. 

As an example we evaluated the community structure of the US-airport dataset m 
under the nulls ME1, ME2, and ME3. This network has a maximum degree of k max = 
312 and a cut-off degree k c = 180 as a consequence, the ME2 will produce an ensemble 
with degree-degree correlations (Fig |4](a) ). ME1 generates correlated ensembles due to 
the conservation of the rich-club, and ME3 uncorrelated ensembles. 

Figure [4] (b)-(d) shows the invariant communities obtained from the different null mod¬ 
els and their links represented with arcs joining them. The top 20 airports are labeled 
in the figure. These 20 airports have a degree greater than the cut-off degree, so the 
communities containing them would be different depending on the null used when defin¬ 
ing them. The relationship between the communities given by the bipartition of the 
network is shown as a dendrogram in the figure. The large dot is the root node of the 
dendrogram. 

The existence of correlations has an effect on the number of invariant communities 
and their composition. The ME1 case, where the ensemble has similar correlations to 
the real network, 15 of the top 20 airports belong to the same community. For the 
ME2 and ME3 ensemble, the top 20 airports are more evenly distributed between the 
communities. 

In particular, LAS , BUR and DEN are always members of the same community inde- 
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Figure 4: (a) The average nearest neighbours degree measured for the data (blue) 

and from the three null models (MEl:green, ME2:brown, ME3:pink). Invariant 
communities obtained using different ensembles, (b) ME1, (c) ME2 and (d) ME3. 
The three letter are the acronyms of the airports, the full name can be found in 
http://www. transtats.bts.gov, 
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pendent of the ensemble used. Their membership to the same community is independent 
of the correlations. Other airports like CLE, DLW and PHL belong to the same com¬ 
munity for the ME3 case. However if there are correlations due to the cut-off degree 
(case ME2) then CLE and DLW will be in the same community and, if we consider a 
correlated case (ME1), then all three airports belong to different communities. 

From this example, it is clear that the existence of correlations in the null models has 
a major impact in defining the communities. What it is not clear is how to assess 
the effect that a change in the null-model’s correlation has in the membership of the 
communities. It is known that a small change, for example the addition of one link, in 
the real network can have a drastic effect on the composition of the communities. For 
instance the resolution limit in community detection [28], where the addition of one link 
between two well defined communities can be interpreted as the existence of a strong 
correlation between these communities and they are merged into one community. In 
this case, the null-model characteristics change very slightly with the new link, so the 
expected number of links between the communities is almost the same with and without 
this link, is the addition of a link in the real network that triggers the merger of the 
communities. To mitigate this behaviour we consider the case where the communities 
are defined using two ensembles. One ensemble captures the relevant properties of the 
real network, in this case the degree sequence and the degree-degree correlations. The 
real network is consider as a possible realisation from this ensemble. The other ensemble, 
which is the null model, has the same degree sequence as the real network but removes 
the degree-degree correlations, so there is no bias in the links connecting the nodes, 
this ensemble is used to evaluate, all things been equal, the expected number of links 
between the communities 


3.1 Soft communities 


If = Lp^j is the expected number of links between nodes i and j when considering 
the null model m, the communities are obtained from the modularity 


N 






JX) _ J2) 


$(9i,9j) 


(7) 


where gi denotes the community that node i belongs, 5(x,y) = 1 if x = y and zero 
otherwise. The term s'X' 1 = Lp\™\ 1 — pf^) is the variance of the mean for the null 
model m and is introduced here as we are comparing the average given by two distribu¬ 
tions. Notice that this normalisation is done between two pair of nodes not between the 
communities. As the constraints defining the ensembles are the averages of the degree 
and rich-club coefficient, we called the communities obtained from the maximisation of 
the modularity, soft-communities. 


Fig. [5](a) shows the communities for the airports network when we compare the average 
number of links between the network with correlations given by ME1, as the correlated 


10 




(a) 



(b) 

Figure 5: Communities defined by ensembles for the Airport network, (a) The network 
has three connected communities. One very densely connected community which con¬ 
tains the top 20 airports, (b) Geographical location of the densely connected community, 
this community also forms the core of the network. 



ensemble and ME2 and ME3 as the decorrelated ensembles. The use of ME2 or ME3 as 
the null gives almost identical communities so the figure shows the case of ME3. The 
communities were obtained using the spectral method. The network has three large 
communities, one of them contains all the top 20 airports. For the rest of the nodes each 
individual node is its own community. Using this ‘smooth’ partition gives a structure 
similar to a core-periphery. The core contains the top 20 airports which are densely 
connected. The periphery is all the other nodes which some form communities and some 
are isolated nodes. For this network the soft communities are not affected by the cut-off 
degree induced correlations. Fig^b) shows the geographical location and links between 
the community that contains the 20 top airports. 

As in the previous example, we evaluated the communities from 100 networks with 
randomised ranks and measure how many times a pair of nodes appear in the same 
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community. There was very little variation on the composition of the communities as a 
function of the ranking order, suggesting that these are stable communities. 


4 Conclusions 

Using the maximal entropy method produces null models that capture the correlation of 
the real networks and also can produce null models with minimal bias. In the case of real 
networks with a cut-off degree greater than the maximum degree, our method removes 
the bias for low degree nodes and minimises the bias for high degree nodes, contrasting 
with link-swapping randomisation methods which introduce correlations at all degrees. 
For the maximal entropy method presented here it is easy to evaluate the cut-off degree 
where the removal of the bias is not possible, giving us a better understanding of the 
null model and hence of its constraints when the null model is used to define network 
structures. 

Representing the network structure via ensembles allow us to define soft communities. 
The communities obtained from the ensembles are more robust to the non-deterministic 
property of the modularity maximisation algorithms and hence the determination of 
the network’s communities. Defining the communities using these ensembles could take 
into consideration that the links describing the connectivity of the network could be 
incomplete or noisy. 


5 Appendix: Algorithm to evaluate the ensemble 


The entropy is 

N N N N 

* = -EE Pi,j log (Pi,j) = -2 EE Pi,j log (ft j) 

j =1 *=1 j =1 i=j +1 

as we are considering undirected networks with no self-loops. Using Eq. @ 
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where F(i) = w(i)(ki - kf), G(i ) = E}Ji F (j), A (*) = Ef= !+ i and = 


Algorithm 


For the null models where the sequence {kf,...,kf} is not given, this sequence is 
obtained by exploring the space of networks, searching for the ensemble with maximal 
entropy. The search is done by choosing two nodes at random, say node i and j, if the 
first node satisfies kf < fc* — 1 and the second node satisfies kf > 0 then we increase kf 

l J l 

by one link and decrease kf by one link. This simple procedure is to satisfy the constraint 

YliLi k f = L/ 2. Every time that a kf is changed the probabilities {p} are evaluated 
using Eqs. 0 and ([2]). The entropy of the network is evaluated from Eq. ©• 


We used Simulated Annealing and a greedy algorithm to find {kf}, we noticed that both 
algorithms gave very similar results, so we present only the greedy algorithm. 

Require: {ki} sorted in decreasing order 

Require: {kf}. If this sequence is not given generate it at random with the condition 
kf < ki 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 


Sold <— S, evaluate the entropy S 
5S ^ 1 

while SS f 0 do 

Select nodes i and j at random, where i f j 
if kf < ki and kf < i — 1 and kf >1 then 

k t <- k t + 1 

kf <- kf - 1 

Snew <— S , evaluate the new entropy 

5S <— S new — S 0 id, evaluate the change in the entropy 

Sold Snew 

if 5S > 0 then 

k t k t - 1 
k t ;•- k t + 1 

end if 
end if 
end while 

{At the end of the while loop we obtain the sequence {kf} which gives the network 
with minimal entropy} 


Lines 6-7 change the connectivity of the rich-club coefficient via kkf , lines 11-14 reject 
the new connectivity if the entropy has not decreased and reset the values of the rich-club 
coefficient. 

For the case that the constraints are kf < ki only line 5 from the above algorithm needs 
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to be modified to 

if kf < ki and kf > 1 then 

L J 

end if 

To evaluate the entropy 
Require: k m and kf for m = 1,... ,N 
1: ie(l) = 1 

2: G'(l) w(l)(ki — kf) 

3: for m = 2 to N do 

4: w(m) <— w(m — 1) * G{m — 1)/( G(m — 1) — kf * w(m)) 

5: G(m) <— G(m — 1) + w(m) * (k m — kf) 

6: end for 
7: B( 1) = 0 
8: for m = N to 2 do 
9: A(m) 4— kf/G(m) 

10: B(m) <— B(m — 1) + (, k m /G(m ) * log (k m /G(m))) 

11: end for 

12: for m = 1 to N do 
13: Cl i — 0 

14: if w(m) * (. k m — kf) f 0 then 

15: a <— w[m) * ( k m — kf)/L * \og{w(m) * (. k m — kf)/L) 

16: end if 

17: S(m) <— S(m — 1) + a * A(m) + w(m) * (k m — kf) * B(m)/L 

18: end for 
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